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(54) Image synthesis system 



(57) An image synthesis system comprises a vol- 
ume memory (12) for accumulating entire volume data 
(32) after being divided into cubic data comprising units 
each of which is composed of eight pieces of volume 
data for constructing a small cube to make it possible to 
perform memory operation by using the units of the cu- 
bic data; an image processing unit (14) including a mem- 
ory control unit (20), an inclination calculation unit (22), 
a brightness calculation unit (24), a synthesis calcula- 
tion unit (26), and a coordinate conversion unit (28); an 



image display memory (1 6); and a display unit (18). The 
volume data is read from the volume memory (12) by 
using the units each of which is composed of eight vox- 
els for constructing the small cube. The inclination cal- 
culation is performed by the inclination calculation unit 
(22), the brightness is calculated by the brightness cal- 
culation unit (24), and the synthesis calculation unit (26) 
is used to perform the synthesis calculation for the entire 
volume data (32) along voxels through which an arbi- 
trarily set imaginary line of sight passes. 
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Description 

[0001] The present invention relates to a volume ren- 
dering technique which plays an important role when nu- 
meric data (volume data) defined in a three-dimensional 
space is visualized on a display screen. In particular the 
present invention relates to a hardware-equipped image 
synthesis system which makes it possible to perform 
high speed drawing processing for three-dimensional 
volume data to be displayed on a display screen. Fur- 
ther, the present invention relates to an image synthesis 
system constructed by using a small amount of hard- 
ware resources. 

Description of the Related Art: 

[0002] The technique, which is used when a three-di- 
mensional model represented by numeric data (volume 
data) defined in a three-dimensional space is visualized 
on a screen of a display unit, is called "volume render- 
ing" Foi example, in the medical field, the volume ren- 
dering technique is utilized when numeric data (volume 
data) concerning the inside of human body, which is ob- 
tained by using a measuring instrument such as CT 
scanner and MR! ic displayed as an image on a display. 
Furthor tho volume rendering technique is also utilized 
in a variety of fields 

[0003] The volume data for representing the three-di- 
mensional mndol is expressed by using various meth- 
ods However there is a method in which expression is 
made by using a sc: of small cubes (voxels) arranged 
on a three-dimensional lattice, in the same manner as 
in the method m when a figure on a two-dimensional 
plane is expressed by using a set of small square picture 
elements (pixels) arranged on two-dimensional lattice 
points 

[0004] The technique for expressing the three-dimen- 
sional model by me«ns of the voxel expression has the 
following features Tnat is. although the amount of data 
is enormous the data structure is simple, in which the 
set operation can be performed extremely easily. The 
technique is not limned to the expression of a three-di- 
mensional model wNch is artificial and regular, but it is 
suitable when an extremely irregular shape existing in 
the nature is expressed 

[0005] One of known rendering techniques based on 
CG (Computer Graphics) is a technique called "ray cast- 
ing method" o/ "ray tiacmg method". In the ray casting 
method, an objective thiee-dimensionai model, which is 
constructed by voxels is arranged in an imaginary 
three-dimensional space defined in a memory space. 
Numeric data, which represents internal information of 
the objective three-dimensional model is given as vol- 
ume data to each of the voxels. According to the ray 
casting method, various pieces of internal information 
concerning the objective three-dimensional model can 
be freely visualized depending on the purpose. There- 
fore, this method attracts attention in recent years as a 



new visualizing technique (volume visualization) for the 
three-dimensional model represented by those for living 
bodies. 

[0006] The volume data as described above includes 

5 data called "field data" on the point (voxel) on the three- 
dimensional lattice. The field data is data, for example, 
for the color value and the transparency. In the case of 
the CT scanner, the density of the tissue in the living 
body is used as the field data. The voxels are colored 

io in accordance with the density to display (visualize) the 
volume data on the screen. Thus, the situation of the 
entire data can be made more comprehensible. 
[0007] The method for visualizing the volume data 
has been hitherto roughly classified into two kinds of 

is methods. The first method is the surface rendering 
method in which voxels having equal data in the volume 
data are connected to one another to form an equivalent 
surface, and primitives such as surfaces and lines are 
allotted to the equivalent surface to make display. The 

20 second method is the method in which the volume data 
is directly displayed by using, for example, the ray cast- 
ing method. 

[0008] When the surface rendering method is used, 
an equivalent surface (appreciated as expansion of the 

25 two-dimensional contour line) is prepared by connecting 
equal volume data. Surfaces and patches are applied 
to the equivalent surface to perform the rendering by 
means of the surface graphics so that the display is 
made. That is, this method is also called "slice-by-slice 

30 method". Japanese Patent Publication (a publication of 
examined application) No. 7-1 20434 shows that the de- 
tails of the method are described in the following docu- 
ment (A). 

[0009] (A) Drebia R. A. , Carpenter, L, and Hanrahan, 
35 p., "Volume Rendering", Computer Graphics, Vol. 22, 
No. 4, pp. 65-74, 1988. 

[0010] When the ray casting method is used, the fol- 
lowing procedure is adopted. That is, rays (lines of sight) 
passing through respective cells are generated from a 

40 point of sight (pixel) on a surface to be displayed (sur- 
face on the display). The^volume data is subjected to 
integration along the rays. Actually, in place of the inte- 
gration, approximate calculation is performed by means 
of numerical integration based on the use of values ob- 

45 tained at several sampling points disposed along the 
rays. 

[0011] The sampling method includes a method in 
which sampling is performed at equal intervals along the 
rays, and a method in which points of intersection be- 

so tween the ray and the cell are used as sampling points 
to make evaluation. The objective data is the volume 
data defined on the orthogonal structural lattice in the 
former case or on the non-structural lattice in the latter 
case. The Japanese Patent Publication No. 7-120434 

55 also shows that the details of the above are described 
in the following documents (B) and (C). 
[0012] (B) Levoy, M., "Display of Surfaces from Vol- 
ume Data", IEEE Computer Graphics and Applications, 
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Vol. 8, No. 3, pp. 29-37, 1988. 

[0013] (C) Garrity, M. P., "Raytracing Irregular Volume 
Data". Computer Graphics, Vol. 24, No. 5, pp. 35-40, 
1990. 

[0014] However, in the present circumstances, when 5 
it is intended to realize a high degree ol display function 
by means of the conventional volume rendering tech- 
nique as described above, it is necessary for any of the 
techniques to use enormous calculation time. 
[001 5] Such a situation results from the following fact. 
That is, in the case of the two-dimensional surface ren- 
dering, exclusive processors have been developed for 
the purpose of high speed processing, and they are 
commercially provided to the market as major constitu- 
tive elements for the engineering work station. As com- 
pared with the two-dimensional surface rendering, no 
hardware-equipped apparatus capable of high speed 
processing has been provided for the three-dimensional 
process, because of the following reasons. 
[0016] Thai is, the following fact is firstly pointed out. 
A large capacity memory is required to accumulate the 
volume data for the three-dimensional space. Further, 
in order to perform the operation processing for the vol- 
ume data in real time, it is necessary to read, at a high 
speed, the data accumulated in the large capacity mem- 
ory. However, neither suitable memory nor appropriate 
interface is available to solve such a problem, resulting 
in an obstacle to realize hardware equipment. 
[0017] Secondly, there has been the following cause. 
That is, in order to realize the three-dimensional visual- 
ization, it is necessary to perform the data sampling from 
the volume data over the entire three-dimensional 
space. When the sampled volume data is subjected to 
operation, a large capacity memory is required to accu- 
mulate intermediate results of the calculation. There- 
fore, it is difficult to realize hardware equipment. Such 
a process has been hitherto realized dominantly by us- 
ing any software. 

[0018] Because of the reasons as described above, 
a lot of time has been hitherto required for the apparatus 
for visualizing the volume data concerning the three-di- 
mensional space. An inconvenience arises in that when 
the point of sight or the objective body is moved in the 
three-dimensional space, it is difficult to synthesize, in 
real time, an image which reflects the movement. 
[0019] When the three-dimensional image is dis- 
played by using the conventional volume rendering 
technique, it is necessary to use enormous volume data 
concerning the three-dimensional space for expressing 
the three-dimensional model. Therefore, it is required to 
accumulate the volume data in a large capacity volume 
memory. When it is intended to deal with the entire 
three-dimensional volume data accommodated in the 
large capacity volume memory at once so that the op- 
eration processing is performed to effect visualization 
(image processing), an inconvenience arises in that it is 
necessary to use a huge amount of hardware resources. 
[0020] For example, when the volume data of respec- 



tive voxels is subjected to the operation along the rays 
by using the ray casting method described above, it is 
a matter of course that a large capacity memory to ac- 
cumulate intermediate results is required as a memory 
to accumulate the intermediate results of the operation. 
The necessity for the hardware resources having the 
large capacity as described above causes prohibition of 
hardware equipment for the image processing unit for 
processing the three-dimensional volume data. 
[0021] A general aim of the present invention is to pro- 
vide a hardware-equipped image synthesis system 
which makes it possible to perform high speed drawing 
for three-dimensional volume data on a display. 
[0022] A principal aim of the present invention is to 
provide an image synthesis system which makes it un- 
necessary to retain intermediate calculation results and 
which makes it possible to make the memory capacity 
necessary and minimum. 

[0023] Another aim of the present invention is to real- 
ize, with a small amount of hardware resources, an im- 
age synthesis system which makes it possible to per- 
form high speed drawing for three-dimensional volume 
data on a display. 

[0024] The above and other aims, features, and ad- 
vantages of the present invention will become more ap- 
parent from the following description when taken in con- 
junction with the accompanying drawings in which a pre- 
ferred embodiment of the present invention is shown by 
way of illustrative example. 

[0025] Embodiments of the present invention will now 
be described by way of example only, with reference to 
the accompanying drawings, in which: - 

FIG. 1 shows a schematic arrangement of an image 
synthesis system concerning a first embodiment to 
carry out the three-dimensional volume rendering 
according to the present invention; 
. FIG. 2A shows the entire volume data, and FIG. 2B 
shows the concept of the voxel based on the unit of 
small cube for constructing the volume data shown 
in FIG. 2A; 

FIG. 3 conceptually shows the volume data accom- 
modated in the volume memory; 
FIG. 4 illustrates the method for processing the vol- 
ume data in accordance with the slice parallel proc- 
ess; 

FIG. 5 illustrates the inclination calculation for the 
volume data performed in the inclination calculation 
unit; 

FIG. 6 A shows an arrangement of the volume data 
comprising units each of which is composed of a 
small cube for the volume data, and FIG. 6B shows 
an array of eight voxels of each small cube shown 
in FIG. 6A; 

FIG . 7 shows an arrangement of the memory control 
unit of the image processing unit; 
FIGS. 8A and 8B show the relationship between a 
central voxel and voxels adjacent thereto, required 
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for the inclination calculation, wherein FIG. 8A 
shows a perspective view, and FIG. 8B shows a 
front view; 

FIGS. 9A to 9H show schematic illustrations depict- 
ing the situation in which the volume data necessary 5 
for the inclination calculation is processed in a slice 
parallel manner; 

FIG. 10 shows a schematic illustration for explain- 
ing the synthesis calculation process in accordance 
with the ray casting method; - to 

FIGS. 11 A and 11 B explain the synthesis calcula- 
tion process performed in the synthesis calculation 
unit, wherein FIG. 11 A shows a perspective view, 
and FIG. 11 B shows a side view; 
FIGS. 1 2A to 1 2D explain the reading of the volume *5 
data in the inclination calculation, wherein FIG! 12A 
shows a back surface, FIG. 12B shows a central 
front side, FIG. 1 2C shows a central back side, and 
FIG. 12D shows a front surface; 

FIG. 1 3 shows a connected arrangement of the vol- 20 
ume memory and the image processing unit (ASIC); 
FIGS. 14A and 14B shows the arrangement of the 
volume data, wherein FIG. 14A illustrate the volume 
data of 32 x 32 x 32, and FIG. 14B illustrates the 
entire volume data; 2s 
FIG. 15 shows an example of reading of the volume 
data when the principal axis is changed; 
FIG. 16 shows the structure of the volume memory 
composed of 512 x 512 x 1024 for accumulating 
the volume data; 30 
FIGS. 17A to 1 7C show an example of the address 
map in the case of the memory structure shown in 
FIG. 16; 

FIG. 18 explains the process of reading of the vol- 
ume data; 35 
FIG. 19 shows a schematic arrangement of an im- 
age synthesis system concerning a second embod- 
iment to carry out the three-dimensional volume 
rendering according to the present invention; 
FIG. 20A shows a top view illustrating the concept 40 
of the volume rendering and the concept of the base 
plane memory, and FIG. 20B shows a side view 
thereof; 

FIG. 21 shows the concept in which the entire vol- 
ume data is divided into a plurality of pieces of vol- <*s 
ume data; 

FIG. 22 shows the base plane in which the entire 
volume data is divided into a plurality of pieces of 
volume data; 

FIG. 23 shows the relationship between the line of so 
sight and the respective voxels in the volume data; 
Fl GS. 24A to 24H show the memory address format 
and the data format used when results of the syn- 
thesis calculation are stored in and read from the 
temporary storage memory; ss 
FIG. 25 A shows the divided volume data obtained 
when the entire volume data is divided into eight 
pieces of divided volume data, and FIG. 25B shows 



the relationship between positions of the respective 
pieces of divided volume data shown in FIG. 25 A 
and coordinate values of u, v, and w; 
FIG. 26 A shows the processing sequence for the 
divided volume data depending on the direction of 
the line of sight in the volume rendering, and FIGS. 
26B to 26E show the processing sequence for the 
divided volume data depending on the difference in 
direction of line of sight; and 
FIG. 27 shows the concept for explaining the read- 
ing control for the memory control unit. 

[0026] At first, reference will be made to a first em- 
bodiment of the image synthesis system according to 
the present invention, and explanation will be made in 
detail below with reference to the accompanying draw- 
ings. 

[0027] FIG. 1 shows a system construction illustrating 
a schematic arrangement of an image synthesis system 
to carry out the three-dimensional volume rendering ac- 
cording to the present invention. 

[0028] The image synthesis system 10 according to 
the embodiment of the present invention comprises a 
volume memory 1 2 constructed by a high speed transfer 
bus memory including a volume data-accumulating unit 
12a for accumulating volume data for representing a 
model in a three-dimensional space and a synthesis cal- 
culation result-accumulating unit 12b for accumulating 
results of synthesis calculation; an image processing 
unit 1 4 including a memory control unit 20, an inclination 
calculation unit 22, a brightness calculation unit 24, a 
synthesis calculation unit 26 : and a coordinate conver- 
sion unit 28; an image display memory 16; a display unit 
18 such as a CRT display; and memory buses 30. 
[0029] The volume memory 1 2 is, for example, a bus 
memory capable of high speed data transfer in conform- 
ity with the specification proposed by Rambus Inc., de- 
tails of which are disclosed in the following document 
(D). 

[0030] (D) Norihiko NAONO, "Direct Rambus aims at 
main memory of personal computer in 1999 (First vol- 
ume), (Latter volume) - , NIKKEI ELECTRONICS, No. 
708, pp. 139-152, 1998, No. 710, pp. 163-176, 1998. 
[0031] The volume memory 12 stores, in the volume 
data-accumulating unit 12a, the volume data for repre- 
senting a model in the three-dimensional space to be 
used as a processing objective, and it stores, in the syn- 
thesis calculation result-accumulating unit 12b, interme- 
diate results of the calculation performed by the synthe- 
sis calculation unit 26 of the image processing unit 14. 
[0032] In the embodiment of the present invention, as 
shown in FIGS. 2Aand2B, e i ght vox els 0 to 7 for forming 
a small cube 34 are used as a unit to define the entire 
volume data 32 in the three-dimensional space. The en- 
tire volume data 32 is stored in the volume data-accu- 
mulating unit 1 2a, for which reading and writing are per- 
formed in conjunction with the image processing unit 14 
in accordance with the control effected by the memory 
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control unit 20. 

[0033] According to this arrangement, the number of 
pieces of volume data which can be read at once is in- 
creased to improve the reading speed. Further, it is pos- 
sible to read, in a small number of times, the data nec- s 
essary for the inclination calculation performed in the 
inclination calculation unit 22 as described later on, 
making it unnecessary to accumulate the previous and 
subsequent volume data which would be otherwise nec- 
essary for the calculation. 

[0034] An example of the volume data for constructing 
the model in the three-dimensional space in the present 
invention is the measurement data obtained by using an 
ultrasonic wave diagnosis apparatus used in the medi- 
cal field. This apparatus is used such that the ultrasonic 
wave is applied to a living body and information con- 
cerning the tissue in the living body is obtained from the 
reflected wave data to be accumulated in a three-dimen- 
sional manner so that an image is displayed to observe 
the situation of the tissue in the living body at various 
angles. The volume data in this example is, for instance, 
the density of the tissue in the living body. One piece of 
volume data (including the respective voxels 0 to 7 
shown in FIG. 2B) is represented, for example, by 8-bit 
numeric data. 

[0035] FIG. 3 conceptually shows the volume data 
stored in the volume memory 12. The voxels of all of the 
small cubes 34 constituted by the volume data, i.e., the 
respective pieces of 8-bit data of the respective eight 
voxels 0 to 7 are stored in the volume data-accumulating 
unit 12a which is connected via the memory bus 30 to 
the image processing unit 14, as indicated by memory 
elements 13a to 13n shown in FIG. 3. 
[0036] The calculation process, which is performed in 
the image processing unit 14 of the embodiment of the 
present invention, is effected by using the slice parallel 
process. The slice parallel process is the method per- 
formed as follows as shown in FIG. 4 for successively 
processing the volume data of the respective small 
cubes 34 for constituting the entire volume data 32, 
starting from the small cubes 34 concerning the forward- 
most surface (right side in FIG. 4). That is, the data cor- 
responding to one surface is processed, and the data 
corresponding to the next one surface is processed after 
completion of the process for the foregoing one surface. 
Thus, the data is successively processed for each sur- 
face in the direction indicated by the arrow A. In this pro- 
cedure, the process for reading the voxel data from the 
volume data-accumulating unit 12a is performed in ac- 
cordance with the control effected by the memory con- 
trol unit 20. 

[0037] In the inclination calculation unit 22, as shown 
in FIG. 5, assuming that the processing objective voxel 
is "m", the inclination calculation for determining the in- 
clination of the image is performed by using the voxel 
"m" as a center, on the basis of the volume data con- 
cerning the voxel "e" disposed in front thereof, the voxel 
T disposed at the back thereof, the voxel "a" disposed 



at the left thereof, the voxel "b° disposed at the right 
thereof, the voxel M d" disposed upwardly, and the voxel 
a c a disposed downwardly. 

[0038] When the data processing is performed in the 
inclination calculation in accordance with the slice par- 
allel process described above, the left and right voxels 
"a", M b tt are disposed just adjacent to the voxel "m" as 
the processing objective. However, the upward and 
downward voxels °d", B c" correspond to the data stored 
previously or subsequently by an amount of one beam 
(one piece in the lateral direction) with respect to the 
voxel "m", and the front and back voxels "e", V corre- 
spond to the data stored previously or subsequently by 
an amount of one slice (one surface) with respect to the 
voxel B m H . Therefore, if the process for reading the vol- 
ume data from the memory is constructed such that the 
data array concerning each flat surface is read one sur- 
face by one surface, it is necessary to store the data 
corresponding to the two surfaces disposed at the front 
and the back, in addition to the data for the surface in- 
cluding the voxel "m" as the processing objective, in or- 
der to perform the inclination calculation performed in 
the inclination calculation unit 22. 
[0039] In the image synthesis system 1 0 according to 
the embodiment of the present invention, the volume da- 
ta is divided to have the data structure in which the small 
cube 34 is used as the unit comprising the eight voxels 
0 to 7. Further, the bus memory capable of high speed 
data transfer is used as the volume memory 1 2. Accord- 
ingly, as understood from the following explanation; it is 
unnecessary to provide the memory for storing the data 
corresponding to the two surfaces disposed at the front 
and the back., in order to perform the inclination calcu- 
lation in accordance with the slice parallel process. 
[0040] FIGS. 6A and 6B show the arrangement of the 
volume data comprising the units of small cubes S1 to 
S6 concerning the entire volume data 32. As shown in 
FIG. 6B, each of the small cubes S1 to S6 shown in FIG. 
6A comprises the unit of eight voxels 0 to 7 for construct- 
ing one small cube 34 : which is arranged continuously 
in the depth direction. The small cubes S1 to S6 are ar- 
ranged and accumulated in the direction of the line of 
sight indicated by the arrow B shown in FIG. 6 A, in the 
volume data-accumulating unit 1 2a of the volume mem- 
ory 12 so that the cubic data (which refers to the data 
comprising the units each of which is composed of the 
eight voxels 0 to 7 for constituting the small cube S1) 
may be read continuously from the entire volume data 
32. 

[0041] That is, the volume data in the embodiment of 
the present invention comprises, for example, 256 piec- 
es in the vertical, lateral, and depth directions respec- 
tively, i.e., 256 x 256 x 256 pieces of volume data in 
total, and it is read by using the unit of the cubic data 
(eight voxels 0 to 7) represented by each of the small 
cubes S1 to S6. 

[0042] FIG. 7 shows an arrangement of the memory 
control unit 20 of the image processing unit 14. The 



75 



20 



25 



30 



35 



40 



45 



SO 



5 



BNSDCCID: < E P 096 1 223A2 J_> 



9 



EP 0 961 223 A2 



10 



memory control unit 20 has registers 70a to 70 e and a 
selector 72. When the volume rendering is started, then 
the cubic data corresponding to one lateral array is firstly 
read from the volume data-accumulating unit 12a of the 
volume memory 12, and the data is stored in any of the 
registers 70a to 70e shown in FIG. 7. 
[0043] The volume data read from the volume mem- 
ory 12 is stored in any of the registers 70a to 70e. The 
destination of the storage is controlled based on the use 
of a general sequencer so that the reading of the volume 
data from the registers 70a to 70e to the inclination cal- 
culation unit 22 is not competed with the writing from the 
volume memory 12 to the registers 70a to 70e. 
[0044] Each of the registers 70a to 70e is able to store 
256 pieces of volume data in the depth direction, i.e., 
the volume data arranged as shown in FIG. 6B. Further, 
the following arrangement is given. That is, the volume 
data of 4 x 256 pieces, which is all of the cubic data 
including the voxels 0 to 7, can be stored in the registers 
70a to 70c. The volume data of 2 x 256 pieces, which 
corresponds to any two of 0, 2, 4, 6 or 1 , 3, 5, 7 disposed 
on one side of the cubic data, can be stored in the reg- 
isters 70d, 70e. 

[0045] In the case of the embodiment of the present 
invention, each of the registers 70a to 70e is constructed 
to store pieces of the volume data of 256 / number of 
ASIC's concerning the actual depth, because the image 
processing unit 14 comprises a plurality of ASIC's, and 
the plurality of ASIC's are simultaneously operated as 
described later on. 

[0046] The data is processed by using the unit of the 
cubic data during the process performed over a range 
from the volume memory 12 in which the volume data 
is accumulated to the memory control unit 20 of the im- 
age processing unit 14. At the stage at which the data 
is read to the memory control unit 20, the cubic data is 
decomposed into the volume data. In the process per- 
formed thereafter, the calculation process is performed 
by using the unit comprising the adjoining voxels nec- 
essary for the inclination calculation on the basis of the 
central voxel "m" in the inclination calculation. The proc- 
ess is successively performed in accordance with the 
slice parallel procedure. 

[0047] That is, when the volume data is read from the 
volume memory 12 via the memory bus 30 (high speed 
data transfer bus), and the data is stored in the registers 
70a to 70e, then the memory control unit 20 selects the 
volume data to be necessary in the inclination calcula- 
tion unit 22 from the selectors 72 shown in FIG. 7 to 
supply the data to the inclination calculation unit 22. 
[0048] In this process, the registers 70a to 70e make 
output by the aid of the selector 72 while making allot- 
ment to the small cube M to serve as the center for the 
inclination calculation, the upward small cube U, the 
downward small cube D, the frontward small cube F, and 
the backward small cube B which are adjacent to the 
small cube M respectively. 

[0049] FIGS. 8 A and 8B show the relationship be- 



tween the central small cube M necessary for the incli- 
nation calculation and the small cubes U, D, R B adja- 
cent thereto. FIG. 8A shows a perspective view, and 
FIG. 8B shows a front view. Adjoining voxels, which are 
s necessary for the calculation when the central voxel in 
the inclination calculation is M0 and M1, are indicated 
by solid circles. R and L indicate the small cubes which 
are disposed on the right and left sides of the central > 
small cube M respectively. 
io [0050] When the central voxel for the inclination cal- 
culation is M0 or M1 , it is necessary for the calculation 
to use the respective volume data concerning the ad- 
joining voxels D2, D3 for the small cube D disposed on 
the downward side of the central voxel, the adjoining 
is voxels F4, F5 for the small cube F disposed on the front 
side, and the adjoining voxels L1 , R0 for the small cube 
L disposed on the left side or the small cube R disposed 
on the right side. When the central voxel for the inclina- 
tion calculation is any one of M2 to M7 other than M0 or 
20 M1 , it is also necessary for the inclination calculation to 
use the respective voxels in the small cubes M, U, D, F, 
B adjacent to each of the central voxels. The necessary 
data is supplied from the respective registers 70a to 70 e 
to the inclination calculation unit 22. 
25 [0051] FIGS. 9A to 9H schematically illustrate the sit- 
uations in which the volume data necessary for the in- 
clination calculation is processed in accordance with the 
slice parallel procedure. FIGS. 9A to 9H are illustrative 
of the case based on the use of the small cubes S1 to 
30 S6 shown in FIGS. 6 A and 6B. and they show front views 
in which the central voxel for the inclination calculation 
and the adjoining voxels necessary for the inclination 
calculation, i.e., the upward, downward, right, and left 
voxels adjacent to the central voxel are depicted by solid 
35 circles. The direction of the inclination calculation proc- 
ess is in the order of FIG. 9A to FIG. 9H. 
[0052] The central voxel for the inclination calculation 
is moved upwardly by every one voxel every time when 
the inclination calculation performed by using the center 
40 is completed for the process for those arranged in the 
depth direction with respect to the plane of paper of 
FIGS. 9Ato9H. According to FIGS. 9A to 9H, it is easily 
understood that the next inclination calculation process 
can be performed by newly reading the cubic data cor- 
45 responding to one lateral array from the volume data : 
accumulating unit 12a every time when the central voxel 
for the inclination calculation process is moved upwardly 
by one. That is, it is demonstrated that the cubic data 
corresponding to the next one lateral array necessary 
50 for the next inclination calculation process is success- 
fully read from the volume data-accumulating unit 12a 
during the period in which the inclination calculation 
process is performed for the data corresponding to the 
previous one lateral array. 
ss [0053] The results of calculation obtained by the incli- 
nation calculation unit 22 are fed to the brightness cal- 
culation unit 24 in which the brightness data correspond- 
ing to the calculation results is calculated to be supplied 
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to the synthesis calculation unit 26. As schematically 
shown in FIG. 10, the volume rendering in accordance 
with the ray casting method is effected such that the vox- 
els 301, which are disposed along the line of sight C 
generated from the point of sight 302 (through which the 
line of sight C passes), are subjected to the synthesis 
calculation in the entire volume data 32. In FIG. 10, open 
circles represent only the voxels 302 through which the, 
line of sight C passes. 

[0054] In the synthesis calculation process : the result 
of brightness calculation is accumulated one by one for 
each of the voxels 301 through which the line of sight C 
passes. The synthesis calculation results, which have 
been obtained for those up to the previous voxel 301, 
are accumulated in the synthesis calculation result-ac- 
cumulating unit 12b. The calculation result, which is nec- 
essary for the synthesis calculation, is read depending 
on the direction of the line of sight C. 
[0055] That is, as shown in a perspective view illus- 
trated in FIG. 11 A, the calculation is started from the 
slice surface which is near to the point of sight 302, in- 
cluded in the entire volume data 32. The surfaces dis- 
posed at the deeper positions are successively subject- 
ed to the calculation. However, the synthesis calculation 
performed in the synthesis calculation unit 26 is the 
process for calculating the accumulation of the voxels 
through which the line of sight 302 has already passed. 
[0056] Therefore, the synthesis calculation unit 26 is 
operated such that if the slice surface for the entire vol- 
l ne data 32 which is being processed is the surface S 
shown in FIG. 11 A, a group of vectors V, which indicate 
the next destination of the synthesis calculation, are de- 
termined in accordance with the result of the accumula- 
tion calculation up to the surface S. As described above, 
in order to perform the synthesis calculation process for 
the certain surface S included in the entire volume data 
32, it is necessary to provide the accumulation calcula- 
tion result up to the previous surface. 
[0057] Accordingly, in the synthesis calculation unit 
26 concerning the embodiment of the present invention, 
the synthesis calculation result of the specified surface 
S included in the entire volume data 32 is stored in the 
synthesis calculation result-accumulating unit 1 2b of the 
volume memory 12. The stored result of the synthesis 
calculation is read in the synthesis calculation process 
for the next surface to successively perform the accu- 
mulation calculation. 

[0058] In this embodiment, the image processing unit 
14 (ASIC), which comprises, for example, the inclination 
calculation unit 22, the brightness calculation unit 24, 
and the synthesis calculation unit 26, has the structure 
capable of the pipeline processing. The processing abil- 
ity of the volume memory 12 is about four times the abil- 
ity of the pipeline processing of the image processing 
unit 14. 

[0059] Accordingly, owing to the pipeline processing 
effected by the image processing unit 14, those which 
can be allotted to the volume memory 12 during the pe- 



riod in which the voxels corresponding to one lateral ar- 
ray of the volume data are processed include three proc- 
esses of (a) reading the next volume data corresponding 
to one lateral array, (b) reading the synthesis calculation 

s result, and (c) save-writing the synthesis calculation re- 
sult. Therefore, even in the case of the structure in which 
the volume data-accumulating unit 12a and the synthe- 
sis calculation result -accumulating unit 12b are exter- 
nally equipped, it is possible to execute the pipeline op- 

io eration of the image processing unit 14 without taking 
any break. 

[0060] In the memory control method according to the 
embodiment of the present invention, as shown in FIGS. 
12A to 1 2D, the same voxel 301 is used in the inclination 

15 calculation four times for the back surface, the central 
front side, the central back side, and the front surface. 
No memory is provided in the image processing unit 14. 
Therefore, it is necessary to read the same data from 
the volume data-accumulating unit 12a of the volume 

20 memory 12 at each time. However, this process can be 
performed owing to the realization of the high speed 
memory cycle by adopting the high speed data transfer 
bus. Thus, it is unnecessary to use the memory in the 
image processing unit 14. 

25 [0061] Next, explanation will be made for details of the 
construction of the entire volume data 32, the volume 
memory 12, and the image processing unit 14 (ASIC). 
[0062] FIG. 13 shows the connected arrangement of 
the volume memory 12 and the image processing unit 

30 14 (ASIC). Two memory buses 30 are connected to the 
respective memory elements 13a to 13n and the ASIC 
0 to ASIC 7 for constructing the image processing unit 
14 respectively. The number of processing pipelines per 
one ASIC is four. Eight ASIC's are arranged to make 

35. simultaneous operation. Therefore, thirty-two types of 
parallel processing are performed, and the memory bus- 
es 30 are operated in parallel as those corresponding 
to sixteen. 

[0063] As described above, the entire volume data 32 
40 comprises 256 individuals in the vertical, lateral, and 
depth directions respectively, i.e., 256 x 256 X 256 vox- 
els in total. The voxels are constructed by 512 (8 x 8 x 
8). pieces of data using a unit of volume 32q of 32 x 32 
x 32 as shown in FIG. 14A, which are accumulated in 
45 the volume memory 12 as the entire volume data 32 
shown in FIG. 14B. 

[0064] With reference to FIG. 1 3, the numerals of 000 
to FF1 in the memory elements 13a to 13n represent 
the volume data of the small cube 34 composed of eight 
50 voxels, i.e., the cubic data. As indicated by the numerals 
of 000 to FF1 , the cubic data is skewed and accumulat- 
ed in the volume memory 12. 

[0065] In this embodiment, the data is skewed at the 
inside of the volume 32q to be accumulated. However, 
55 the data is not skewed for the volume group 32q for con- 
structing the entire volume data 32 of 256 X 256 x 256. 
The eight of ASIC 0 to ASIC 7 for constructing the image 
processing unit 14 are operated such that 32 voxels are 
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simultaneously read and processed in the order starting 
from the lower-left position of the volume data. There- 
fore, the read data is skewed. However, the reading ad- 
dress during this process is the same for all ASIC's. 
[0066] It is assumed that the data (Data) is represent- 
ed by a combination of ix, iy, iz corresponding to the re- 
spective low order 4-bit addresses, i.e., Data = [ix, iy, iz] 
provided that the coordinates of the original volume data 
are x, y, z. The memory bus 30 of the sixteen memory 
buses 30, to which a certain piece of data is accommo- 
dated, is determined by the following expression provid- 
ed that the memory bus channel number is k. 

k= (x+y+z)mod1 6 (remainder obtained by dividing 
"x+y+z u by 16) 

[0067] The addresses of the memory bus channels, 
which are given to the respective sixteen memory buses 
depending on the direction of the principal axis when the 
volume rendering is executed, are represented by the 
following expressions. 

Z principal axis: iz = Z, iy = Y, ix = (k-z-y+32)mod1 6 
Y principal axis: iz = (k-x-y+32)mod1 6, iy = Y, ix = X 
X principal axis: iz = Z, iy = (k-x-z+32)mod16, ix = 

X 

[0068] As described above, even when the principal 
axis is changed, it is possible to simultaneously obtain 
the volume data aligned in the lateral direction. The 
reading of the volume data when the principal axis is 
changed is exemplarily shown in FIG. 15. In FIG. 15, k 
represents the memory bus channel number of the six- 
teen memory buses 30. 

[0069] The calculation result, which is obtained by the 
synthesis calculation performed by the synthesis calcu- 
lation unit 26 as described above, is subjected to the 
coordinate conversion performed by the coordinate con- 
version unit 28 to make output to the image display 
memory 16. The display unit 18 can display an objective 
image on its display section in accordance with the im- 
age data accumulated in the image display memory 16. 
[0070] Next, as shown in FIG. 16, the volume memory 
12 for accumulating the entire volume data 32 is allowed 
to have a structure of 51 2 X 51 2 X 1024. Two channels 
of the memory buses 30 are connected to eight parallel 
ASIC's. Six memories (DRAM) of 18 Mbit are connected 
to the respective memory bus channels to accumulate 
the entire volume data 32. In this case, the address map 
is exemplarily illustrated as shown in FIGS. 1 7A to 1 7C. 
[0071] In FIGS. 17Ato 17C, respective addresses are 
represented by CS for the chip select for DRAM, B for 
the bank select in DRAM, C for the column, R for the 
row, PS for the selection of ASIC, and P for the data 
selection in the packet, and X, Y. Z represent the coor- 
dinate system for the volume data. 
[0072] In the case of the embodiment of the present 
invention, the volume data comprising the units of the 
cubic data is not skewed as described above. Therefore, 
as shown in FIG. 1 8, when the principal axis is changed 
depending on the direction of the line of sight during the 
reading process tor the volume data, it is necessary to 



perform the process for rotating the small cube 34 to 
change the correspondence of the data read to the 
memory control unit 20 of the image processing unit 1 4. 
[0073] Next, a second embodiment of the image syn- 
s thesis system according to the present invention will be 
explained. 

[0074] FIG. 19 schematically shows an arrangement 
of an image synthesis system 40 for carrying out the 
three-dimensional volume rendering according to the 

10 second embodiment. 

[0075] The image synthesis system 40 according to 
this embodiment comprises an image processing unit 
29 including a base plane memory 27 arranged between 
the synthesis calculation unit 26 and the coordinate con- 

is version unit 28 of the image processing unit 14 shown 
in FIG. 1. The other constitutive components or parts 
are the same as those of the image synthesis system 
10 shown in FIG. 1. 

[0076] As explained in the first embodiment, when the 
20 ray casting method is used, the following method is 
adopted. That is, the ray (line of sight), which passes 
through the respective cells, is generated from the point 
of sight (pixel) on the display screen (projection surface) 
of the display, and the data of the voxels for constructing 
25 the volume data is integrated along the ray. 

[0077] FIGS. 20A and 20B show the processing con- 
cept in which the volume data 32 is visualized in accord- 
ance with the method as described above. FIG. 20 A 
shows a top view in which the parallel projection from 
30 the projection surface P is viewed from an upward po- 
sition. FIG. 20B shows a side view in which the parallel 
projection from the projection surface P is viewed from 
a side position. 

[0078] The inclination calculation, the brightness cal- 
35 culation, and the synthesis calculation are successively 
performed by the aid of the image processing unit 29 on 
the basis of the data of the respective voxels (respective 
voxels through which the line of sight C passes) in the 
volume data 32 along the line of sight C extending from 
40 the projection surface P. In the parallel projection, the 
line of sight C cannot proceed any more at the three sur- 
faces as the exits of the line of sight C (the upper surface 
3a, the side surface 3b, and the deep surface 3c in the 
example shown in FIGS. 20A and 20B, the surfaces are 
45 being referred to as "base planes" 3a to 3c). The syn- 
thesis calculation results are accumulated in the base 
plane memory 27. 

[0079] The coordinate conversion unit 28 reads the 
contents of the base plane memory 27 to perform the 

so coordinate conversion into those corresponding to the 
projection surface P. During this process, the coordinate 
conversion unit 28 determines the data of the synthesis 
calculation results for the corresponding places in the 
base plane memory 27 by following the line of sight C 

55 from the side of the projection surface P. Therefore, the 
coordinate conversion process cannot be started unless 
the synthesis calculation is completed for all of the 
points on the base plane, and the accumulation of the 



8 



BNSDOCID: <EP 0961223A2J_> 



15 EP 0 961 223 A2 16 



calculation results into the base plane memory 27 is 
completed 

[0080] Thcrctorc if is necessary lor the base plane 
memory 27 to hnvc .m enough capacity to accumulate 
the data ol all ol the syntrcsis calculation results con- 
cerning the three surfaces (tnc upper surface 3a, the 
side surface 3b and the deep surface 3c). It is indispen- 
sable to use the base plane memory 27 having a capac- 
ity capable of performing operation processing while 
dealing with a lar:jc amount of volume data at once. 
[0081 ] When it is considered in this process to realize 
the performance ol the parallel concurrent operation of 
the coordinate cenvcrson process and the process up 
to the execution of the synthesis calculation in the image 
processing unci 29 after reading the volume data from 
the volume memory 12 it is necessary that the base 
plane memory 2/ is composed ol a dual system (double 
buffer system) 

[0082] For example when the rendering process is 
performed for the vo>urrc cala 32 comprising the voxels 
having one side of 512 ol the three-dimensional lattice, 
it is necessary for the base plane memory 27 to use the 
memory having a capacity of 1 .5 MB provided that the 
double buffer system to Drovidcd because of 51 2 X 51 2 
X 3 surfaces and the c bit voxels to be subjected to the 
synthesis calculation 

[0083] The base plane memory 27 having this capac- 
ity is to be mounted m the image processing unit 29. 
However, when it is intended to construct the image 
proces; >rig unit 29 by using the integrated circuits 
(ASIC), the provision of such a capacity is difficult for 
the present technical level considering, for example, the 
relationship with respect to the memory control unit 20, 
the inclination calculation unit 22, the brightness calcu- 
lation unit 24, and the synthesis calculation unit 26 which 
are other circuits in ASIC 

[0084] Accordingly m the embodiment of the present 
invention, as shown in FlG 21. the system is construct- 
ed such that the volume data 32 is divided into a plurality 
of pieces of volume data i e . eight pieces of volume 
data 32a to 32h in this embodiment to perform the op- 
eration processing for the rendering. In FIG. 21 , symbols 
ab to gh indicate boundary surfaces of the divided piec- 
es of volume data 32a to 32h respectively. 
[0085] FIG. 22 shows the base planes 3a, 3b, 3c ob- 
tained when the volume data 32 is divided into the eight 
pieces of volume data 32a to 32h. When viewed in the 
direction of the line of s»;yh; C. the base plane 3b corre- 
sponding to the lateral dnection is defined to be in the 
n u u direction, the base plane 3a corresponding to the 
vertical direction is defined to be in the *v" direction, and 
the base plane 3c corresponding to the depth direction 
is defined to be in the 'w" direction. 
[0086] The size of tho baso plane memory 27 can be 
decreased by dividing the entire volume data 32 into the 
plurality of pieces of volume data 32a to 32h as de- 
scribed above. That is as shown in FIG. 21, when the 
volume data 32 is divided into the eight pieces, each of 



the volume data 32a to 32h comprises the voxel having 
one side of 256, giving 256 X 256 X 3 surfaces with the 
8-bit voxel to be subjected to the synthesis calculation. 
Assuming that the double buffer system is provided, the 
s capacity is 384 KB, and hence it is possible to make pro- 
vision on ASIC. 

[0087] When the entire volume data 32 is divided into 
the plurality of pieces to perform the visualization proc- 
ess, the process for the boundary surfaces ab to gh may 
cause a problem. As described above, in the volume 
rendering method, the voxels are followed along the line 
of sight C, and the brightness calculation values of the 
followed voxels are subjected to the synthesis calcula- 
tion. Therefore, if the volume data 32 is divided, the syn- 
thesis calculation, which follows the line of sight C, is 
once intercepted by the boundary surfaces ab to gh. 
[0088] Accordingly, in the embodiment of the present 
invention, the synthesis calculation results up to the 
boundary surfaces ab to gh, which are obtained for the 
rendering process for the specified divided volume data, 
are temporarily saved in the memory. When the synthe- 
sis calculation is started for the next divided volume data 
disposed ahead of the boundary surface ab to gh, the 
synthesis calculation results having been saved in the 
memory are read to be used as the initial value for the 
synthesis calculation for performing the rendering proc- 
ess. Accordingly, it is possible to obtain the continuity of 
the synthesis calculation at the boundary surfaces ab to 
gh when the volume data 32 is divided. 
[0089] The memory for temporarily saving the synthe- 
sis calculation results up to the boundary surfaces ab to 
gh is the synthesis calculation result-accumulating unit 
12b of the volume memory 12 as the high speed data 
transfer bus memory. It is considered that no circuit el- 
ement is increased in the image processing unit 14. 
[0090] FIG. 23 shows the relationship between the 
line of sight C generated from the point of sight 302 and 
the respective voxels 301 in the volume data 32. With 
reference to FIG. 23, the line of sight C generated from 
the point of sight 302 enters the divided volume data 
32a. The synthesis calculation is firstly performed in the 
volume data 32a successively along the line of sight C. 
In FIG. 23, the voxel 301 indicated by the solid circle 
indicates the position of abutment against the boundary 
surface ae. The synthesis calculation result up to the 
voxel 301 is accumulated as the synthesis calculation 
result up to the boundary surface ae, in the synthesis 
calculation result-accumulating unit 12b. 
[0091] After that, when the rendering process is per- 
formed for the divided volume- data 32e, the synthesis 
calculation is started from the voxel 301 indicated by the 
double circle. The synthesis calculation result up to the 
voxel 301 indicated by the solid circle is read as the syn- 
thesis calculation result up to the position described 
above from the synthesis calculation result-accumulat- 
ing unit 12b, and it is given as the initial value for the 
synthesis calculation result of the voxel 301 indicated 
by the double circle. Subsequently, the rendering prOC- 
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ess for the volume data 32e is continuously performed 
along the line of sight C. Upon abutment against the 
base plane 3a. the synthesis calculation result is written 
into the base plane memory 27, and the calculation is 
completed. 

[0092] Next, explanation will be made for details of the 
operation of the image synthesis system 40 according 
to the embodiment of the present invention. The proc- 
esses, which are executed up to the brightness calcu- 
lation after reading the volume data 32 from the volume 
memory 1 2, are the same as those performed in the first 
embodiment, explanation of which will be omitted. 
[0093] In this embodiment, the synthesis calculation 
unit 26 does not deal with the entire volume data 32 as 
the operation objective at once. The synthesis calcula- 
tion process is performed by using the units of the divid- 
ed pieces of volume data 32a to 32h. The process is 
successively performed for the respective pieces of di- 
vided volume data 32a to 32h. 

[0094] The process is performed in the synthesis cal- 
culation unit 26 to accumulate, one by one, the bright- 
ness calculation result for each of the voxels 301 
through which the line of sight C passes, concerning the 
divided volume data 32a (see FIG. 23). The synthesis 
calculation results up to the previous voxel 301 are ac- 
cumulated in the synthesis calculation result-accumu- 
lating unit 12b. The calculation result necessary for the 
synthesis calculation is read depending on the direction 
of the line of sight C to successively perform the cumu- 
lative calculation. The synthesis calculation result con- 
cerning the volume data 32a is stored in the base plane 
memory 27, and it is temporarily stored in the synthesis 
calculation result-accumulating unit 12b to be used as 
the initial value when the synthesis calculation is per- 
formed for the next adjoining divided volume data 32e 
(see FIG. 23). 

[0095] When the synthesis calculation is completed 
for the divided volume data 32a, the synthesis calcula- 
tion unit 26 performs the synthesis calculation for the 
next adjoining divided volume data 32e. During this 
process, the synthesis calculation result temporarily 
stored in the synthesis calculation result-accumulating 
unit 12b, i.e., the synthesis calculation result concerning 
the previous divided volume data 32e is read, and it is 
used as the initial value for the synthesis calculation 
concerning the divided volume data 32e. 
[0096] The calculation process is repeated for the re- 
spective pieces of divided volume data 32a to 32h. The 
final synthesis calculation result is stored in the base 
plane memory 27. 

[0097] The synthesis calculation result, which has 
been stored in the base plane memory 27 as described 
above, is subjected to the coordinate conversion per- 
formed by the coordinate conversion unit 28 to make 
output to the image display memory 16. The display unit 
18 can display an objective image on its display section 
in accordance with the image data accumulated in the 
image display memory 16. 



[0098] Next, in the image synthesis system 40 ac- 
cording to the embodiment of the present invention, the 
entire volume data 32, which is composed of 512 x 512 
x 512 (hereinafter referred to as "51 2 3 ") of voxels, is 

5 divided into eight pieces to give pieces of divided volume 
data 32a to 32h (see FIG. 21 ) of 256 X 256 x 256 (here- 
inafter referred to as "256 3 ") in order to perform the ren- 
dering process., Explanation will be made concerning 
this process for the memory address format and the data 

io format used when the synthesis calculation results up 
to the boundary surfaces ab to gh of the respective piec- 
es of divided volume data 32a to 32h are stored in or 
read from the temporary storage memory (upon the use 
of 256 X 64-bit memory). 

is [0099] As shown in FIG. 24A, the address format is 
composed of 18 bits. The bits of 0 to 7 represent the 
column address : the bits of 8 to 16 represent the row 
address, and the bit of 17 represents the bank address. 
As shown in FIG. 24B, the data format is composed of 

20 64 bits including the two sets of data, i.e., the color data 
Co of 1 6 bits and the opacity data a of 1 6 bits. The opac- 
ity data a is the data representing the rat b of existence 
of the color data Co or the ratio of participation exerted 
on other colors. 

2S [01 00] The lower order 8 bits of the respective pieces 
of 1 6-bit data are the decimal part. The data necessary 
for the final image is the high order 8 bits. However, the 
16-bit data containing the low order 8 bits is retained 
during the process of the synthesis calculation in order 

30 to ensure the accuracy of the high order 8 bits when the 
product sum calculation is performed in a cumulative 
manner in the synthesis calculation. 
[0101] In the embodiment of the present invention, the 
rendering can be performed while changing the size of 

3S the view volume. There are two cases, i.e., a case in 
which the memory of 51 2 3 is used as one view volume, 
and a case in which the memory of 51 3 3 is divided into 
eight view volumes of 256 3 to use different pieces of da- 
ta respectively. The former case is referred to as "512 

40 mode", and the latter case is referred to as "256 mode". 
In this process, the rendering of 256 3 is basic, and the 
rendering of 51 2 s is realized by 8 times execution. 
[01 02] In the 51 2 mode, the respective boundary val- 
ues for u, v, w (see FIG. 21) are read and written de- 

45 pending on the line of sight C and the volume data po- 
sition. However, when the rendering ot 256 3 is per- 
formed, it is necessary to accumulate the synthesis cal- 
culation result for one surface concerning the w direction 
in order to perform the synthesis calculation in conjunc- 

so tion with the next surface. The synthesis calculation re- 
sult corresponding to the one surface is referred to as 
"non-boundary". 

[0103] In order to store the synthesis calculation re- 
sults for the respective boundaries in the temporary stor- 
es age memory, the following addresses are required. That 
is, at the "u" boundary, "u" itself is constant, and hence 
the addresses of v, w are required. At the "v B boundary, 
"v B itself is constant, and hence the addresses of u, w 
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are required. At the "w" boundary, "w" itself is constant, 
and hence the addresses of u, v are required. 
[0104] FIG. 24C shows the memory address format 
for the "w" boundary and the non-boundary in the case 
of the 256 mode and the 51 2 mode. FIG. 24D shows the s 
data format corresponding thereto. The ninth and eighth 
bits of the address format represent the mode of use. 
When the two bits are 0. 0, they indicate the "w" bound- 
ary and the non-boundary in the case of the 256 mode 
and the 512 mode. io 
[0105] FIG. 24E shows the memory address format 
for the "u" boundary in the case of the 512 mode. FIG. 
24F shows the data format corresponding thereto. 
When the ninth and eighth bits of the address format are 
0. 1. they indicate the 'u M boundary in the case of the is 
512 mode Similarly, FIG. 24G shows the memory ad- 
dress format tor the V boundary in the case of the 512 
mode FIG 24H shows the data format corresponding 
thereto When the ninth and eighth bits of the address 
format nre l 0. ihey indicate the "v" boundary in the 20 
Cc»s»e of the 512 mode. 

[0106] As for the image synthesis system composed 
of the 32 parallel pipeline system shown in FIG. 13, the 
process is simultaneously performed in 32 parallel man- 
ner m the "u" direction, as viewed in the coordinate sys- 2S 
\om of u v w In the address format described above, 
u4-0 is not allotted Further, two pieces of data are han- 
dled at once by using one memory bus 30. Therefore, 
t\*o pieces of rlata indicated by 0, 1 in the data format 
are read and wril-on 30 
[0107] Next explanation will be made for the se- 
quence of rendering for the divided volume data 32a to 
32h FIG 25A shows the volume data comprising the 
divided pieces of volume data 32a to 32h of 256 X 256 
x 255 obtained by drvidmg. into eight, the entire volume 35 
data 32 composed of the voxels of 51 2x512 X512. In 
FIG 25A numerals® to® are numbers affixed to the 
respectrve pieces of drvided volume data 32a to 32h for 
the purpose of the following explanation. The positions 
of the respectrve pieces of divided volume data® to® 40 
and the u v w coordinate values are shown in FIG. 25B. 
[0108] FIG 26A snows the processing sequence for 
the drvided volume oata®to@depending on the direc- 
tion of the line oi sight C (see FIG. 21) in the volume 
rendering The reason why the processing sequence 45 
has to be changed depending on the direction of the line 
of sight C is that the rendering is performed by succes- 
sively following the voxels along the direction of the line 
of sight C. In othei words, the rendering for the drvided 
volume data disposed far along the line of sight C cannot so 
be performed prior to the rendering for the divided vol- 
ume data disposed at the nearer position. FIGS. 26B to 
26E show, with numerals of 1 to 8, the processing se- 
quence for the divided volume data® to ©respectively 
when the direction of the line of sight is the illustrated ss 
direction of the line of sight C. 

[01 09] When the volume data is divided to perform the 
rendering as described above, it is necessary for the in- 



clination calculation unit 22 to use the voxels at the por- 
tions protruding from the boundary upwardly, down- 
wardly, rightwardly, leftwardly, frontwardly, and back- 
wardly, because of the following reason. That is, as 
shown in FIG. 27, when the right end mr of the divided 
volume data 32a during the rendering is the central vox- 
el for the inclination calculation, the process is per- 
formed on the basis of the voxels disposed upwardly, 
downwardly, rightwardly, leftwardly, frontwardly, and 
backwardly (see FIG. 5) while interposing the central 
voxel therebetween. 

[0110] Accordingly, the memory control unit 20, which 
supplies the volume data to the inclination calculation 
unit 22, excessively reads the volume data while ex- 
ceeding the region of the divided volume data in the ren- 
dering performed in the divided manner. For example, 
the following control is made. That is, when the render- 
ing direction is the "u" direction, the volume data of the 
adjoining left end or right end is excessively read at the 
right end or left end. In the case of the "v" direction, the 
volume data is excessively read in an amount of upper 
or lower one line for each surface to be processed. In 
the case of the "w" direction, the volume data is exces- 
sively read in an amount of one surface at the start or 
end of the process. 



Claims 

1. An image synthesis system comprising: 

a volume memory (12) for accumulating a plu- 
rality of pieces of divided volume data obtained 
, by dividing volume data composed of a plurality 
of voxels defined by numeric data in a three- 
dimensional space; 
an image processing unit (14, 29); 
a memory bus (30) for connecting said volume 
memory (1 2) to said image processing unit (1 4, 
29); 

an image display memory (1 6) for storing image 
data outputted from said image processing unit 
(14); and 

a display unit (18) for displaying said volume 
data on the basis of said image data stored in 
said image display memory (16), wherein said 
image processing unit (14, 29) includes: 
a memory control unit (20) for reading said vol- 
ume data comprising units each of which is 
composed of eight pieces of volume data for 
constructing a small cube, from said volume 
memory (1 2); 

an inclination calculation unit (22) for perform- 
ing inclination calculation on the basis of vol- 
ume data of a central voxel and volume data of 
voxels adjacent to said central voxel, of said 
volume data read by said memory control unit 
(20); 
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a brightness calculation unit (24) tor performing 
brightness calculation on the basis of a calcu- 
lation result obtained by said inclination calcu- 
lation unit (22): 

a synthesis calculation unit (26) for performing s 
synthesis calculation on the basis of a calcula- 
tion result obtained by said brightness calcula- 
tion unit (24) along voxels through which an im- 
aginary line of sight arbitrarily set for said vol- 
ume data passes; and io 
a coordinate conversbn unit (23) for performing 
coordinate conversion for a calculation result 
obtained by said synthesis calculation unit (26). 

The image synthesis system according to claim 1, is 
wherein said volume memory (12) is operated such 
that said volume data is divided into cubic data com- 
prising units each of which is composed of eight 
pieces of volume data for constructing a small cube, 
and said volume data is accumulated by using said 20 
units of cubic data to make it possible to perform 
memory operation. 

The image synthesis system according to claim 1 , 
wherein said synthesis calculation unit (26) per- 2s 
forms said synthesis calculation by using units of 
divided volume data, and said image processing 
unit (29) includes a base plane memory (27) for ac- 
cumulating said calculation result of said synthesis 
calculate unit (26). 30 

The image synthesis system according to claim 1, 
wherein said memory control unit (20) reads a plu- 
rality of pieces of cubic data corresponding to one 
array stored continuously in said volume memory 35 
(1 2), and it includes a plurality of first registers (70a 
to 70c) for storing said read pieces of cubic data 
corresponding to one array as they are, and a plu- 
rality of second registers (70d, 70e) for storing said 
volume data corresponding to one array for con- *o 
structing one surface of said cubic data. 

The image synthesis system according to claim 4, 
wherein said memory control unit (20) includes a 
selector (72) for selecting volume data of a central *s 
voxel for an inclination calculation process per- 
formed by said inclination calculation unit (22) and 
voxels adjacent to said central voxel from said vol- 
ume data stored in said plurality of first registers 
(70a to 70c) and said plurality of second registers so 
(70d, 70e) to supply said selected volume data to 
said inclination calculation unit (22). 

The image synthesis system according to claim 4, 
wherein said memory control unit (20) includes a 55 
selector (72) for selecting cubic data to serve as a 
center for said inclination calculation performed by 
said inclination calculation unit (22) as well as cubic 



data adjoining upwardly, cubic data adjoining down- 
wardly, cubic data adjoining frontwardly and cubic 
data adjoining backwardly with respect to said cubic 
data to serve as said center, from said cubic data 
stored said first registers (70a to 70c) and said sec- 
ond registers (70d, 70e) to supply said selected cu- 
bic data to said inclination calculation unit (22). 

7. The image synthesis system according to claim 4, 
wherein each of said first registers (70a to 70c) and 
said second registers (70d, 70e) has a capacity to 
accumulate volume data corresponding to one ar- 
ray in a depth direction, volume data corresponding 
to one array in a vertical direction, or volume data 
corresponding to one array in a lateral direction ac- 
cumulated continuously in said depth direction, said 
vertical direction, or said lateral direction of said vol- 
ume memory (12). 

8. The image synthesis system according to claim 7, 
wherein said first registers (70a to 70c) comprise at 
least three registers, and said second registers 
(70d, 70e) comprise at least two registers. 

9. The image synthesis system according to claim 7, 
wherein said memory control unit (20) reads cubic 
data corresponding to one lateral array necessary 
for the next step from said volume memory (12) eve- 
ry time when said voxel to serve said center for said 
inclination calculation performed by said inclination 
calculation unit (22) is moved upwardly by one in 
said cubic data 

10. The image synthesis system according to claim 1, 
wherein said memory control unit (20) reads said 
volume data from said volume memory (12) by us- 
ing units of cubic data, and said inclination calcula- 
tion unit (22), said brightness calculation unit (24), 
and said synthesis calculation unit (26) decompose 
said cubic data into pieces of volume data of single 
voxel to perform a calculation process. 

11. The image synthesis system according to claim 1, 
wherein said image processing unit (14, 29) com- 
prises a plurality of elements (ASIC 0 to ASIC 7) for 
making parallel operation, each of said elements 
(ASIC 0 to 7) is connected to a plurality of memory 
elements (1 3a to 13n) for constructing said volume 
memory (12) via said plurality of memory buses 
(30), and said volume data is stored in said volume 
memory, as having skew corresponding to a 
number of said memory buses (30). 

12. The image synthesis system according to claim 1 , 
wherein said volume memory (1 2) comprises a tem- 
porary storage memory (1 2b) for temporarily storing 
said synthesis calculation result concerning said di- 
vided volume data, and said synthesis calculation 
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result stored in said temporary storage memory 
(12b) is read to be used as an initial value for said 
synthesis calculation when said synthesis calcula- 
tion is started for adjacent divided volume data. 

5 

1 3. The image synthesis system according to claim 1 2, 
wherein said temporary storage memory (12b) is 
connected via said memory bus (30) to said image 
processing unit (14). 

10 

14. The image synthesis system according to claim 1, 
wherein said volume memory (12) successively ac- 
cumulates cubic data comprising units each of. 
which is composed of eight pieces of volume data, 
continuously in a depth direction, a vertical direc- is 
tion, and a lateral direction thereof. 

15. The image synthesis system according to claim 1, 
wherein said image processing unit (14, 29) proc- 
esses respective pieces of cubic data for construct- 20 
ing said entire volume data, successively one sur- 
face by one surface starting from cubic data con- 
cerning a forwardmost surface. 

16. The image synthesis system according to claim 1, 2S 
wherein a voxel as a processing objective is used 

as a center, and said inclination calculation unit (22) 
calculates inclination of said voxel as said process- 
ing objective on the basis of a voxel disposed in 
front of said voxel as said processing objective, a 30 
voxel disposed at the back thereof, a voxel dis- 
posed at the left thereof, a voxel disposed at the 
right thereof, a voxel disposed upwardly, and a vox- 
el disposed downwardly. 

35 

17. The image synthesis system according to claim 1, 
wherein said synthesis calculation unit (26) cumu- 
latively calculates brightness data relative to said 
brightness calculation unit (24) concerning voxels 
through which an imaginary line of sight generated 40 
from an arbitrarily set point of sight passes, by read- 
ing said synthesis calculation result concerning a 
previous voxel accumulated in said volume memory 
(12) to cumulatively calculate said brightness data 
relative to said brightness calculation unit (24) so *s 
that said data is accumulated in said volume mem- 
ory (12). 

18. The image synthesis system according to claim 1, 
wherein said volume data is measurement data ob- 50 
tained by using an ultrasonic wave diagnosis appa- 
ratus. 

1 9. The image synthesis system according to claim 1 8, 
wherein said measurement data is density data of ss 
tissue in a living body. 

20. The image synthesis system according to claim 1, 



wherein said volume memory (1 2) is constructed to 
be capable of connection to a high speed data 
transfer bus (30), and it is connected to said image 
processing unit (14) by using said high speed data 
transfer bus (30). 

21. The image synthesis system according to claim 1 , 
wherein said image processing unit (14, 29) com- 
prises a plurality of ASIC's, and each image 
processing unit (14, 29) is constructed to perform 
pipeline processing for processing of said volume 
data corresponding to one lateral array, reading of 
subsequent volume data corresponding to one lat- 
eral array from said volume memory, reading of said 
synthesis calculation result from said synthesis cal- 
culation result -accumulating unit (12b), and 
processing of storage of said synthesis calculation 
result in said synthesis calculation result-accumu- 
lating unit (12b). 

22. The image synthesis system according to claim 21 , 
wherein said image processing unit (14, 29) com- 
prises eight ASIC's, and each of them is connected 
to said volume memory (12) by using two high 
speed data transfer buses (30). 

23. The image synthesis system according to claim 1 , 
wherein said volume data is divided into eight piec- 
es of divided volume data. 

24. The image synthesis system according to claim 1 , 
wherein said image processing unit (14, 29) has a 
first processing mode for performing calculation 
while using said entire volume data as one view vol- 
ume, and a second processing mode for performing 
calculation while using said divided volume data as 
view volumes. 

25. The image synthesis system according to claim 24, 
wherein said first processing mode and said second 
processing mode are determined by a specified bit 
in address information used when said synthesis 
calculation result relative to said synthesis calcula- 
tion unit (26) is written into or read from said volume 
memory (1 2) for temporary storage. 

26. The image synthesis system according to claim 1 , 
wherein said image processing unit (14, 29) chang- 
es a calculation process sequence of said divided 
volume data depending on a direction of an imagi- 
nary line of sight generated from an arbitrarily set 
point of sight. 

27. The image synthesis system according to claim 1 , 
wherein when a central voxel for said inclination cal- 
culation performed in said inclination calculation 
unit (22) is a voxel disposed at an outermost end 
contacting with a boundary of said divided volume 
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data, said memory control unit (20) reads, from said 
volume memory (12), said volume data of a voxel 
adjacent to said central voxel for said inclination cal- 
culation, at an outermost end of adjacent divided 
volume data with said boundary intervening there- 
between. 
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FIG. 2A 
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FIG. 3 
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FIG. 4 
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